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Art Unit: 2625 

DETAILED ACTION 
Double Patenting 

1 . The nonstatutory double patenting rejection is based on a judicially created 
doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the 
unjustified or improper timewise extension of the "right to exclude" granted by a patent 
and to prevent possible harassment by multiple assignees. See In re Goodman, 1 1 
F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 

USPQ 645 (Fed. Cir. 1985); In re Van Ornum. 686 F.2d 937, 214 USPQ 761 (CCPA 
1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970);and, In re Thorington, 
418 F.2d 528, 163 USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1 .3218 may be used 
to overcome an actual or provisional rejection based on a nonstatutory double patenting 
ground provided the conflicting application or patent is shown to be commonly owned 
with this application. See 37 CFR 1 .130(b). 

Effective January 1 , 1994, a registered attorney or agent of record may sign a 

terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 
37 CFR 3.73(b). 

2. Claims 1 -50 rejected under the judicially created doctrine of obviousness-type 
double patenting as being unpatentable overclaims 1-28, of U.S. Patent No. 6,625,299. 
Each of the limitation set forth in the claims of the instant application is defined in the 
claims of the patent. 

For example consider claim 1 , of current application, compared to claim 1 , of 
patent application, it disclose a method for tracking the position and orientation of an 
object, comprising the steps of: 

(a) scanning across an object to detect fiducials, and form runs (column 13 lines 
38-41); 

(b) clumping video runs to detect a pattern of fiducials wherein the step of 
clumping video runs (column 13, lines 42-43); 
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(c) acquiring estimated values for a set of tracking parameters by comparing a 
detected pattern of fiducials to a reference pattern of fiducials (column 13, lines 47-53); 

(d) iterating the estimated values for the set of tracking parameters until the 
detected pattern of fiducials match the reference pattern of fiducials to within a desired 
convergence (column 13, lines 63-65). 

Claim 7, of current application, compared to claim 2, of patent application, it 
disclose a method for tracking the position and orientation of an object of claim 1 , 
wherein the detected and reference patterns of fiducials includes a pseudo random 
pattern of soft fiducials and wherein the step of acquiring estimated values for a set of 
tracking parameters includes the step of electing at least one of the soft fiducials with 
the reference pattern of fiducials to estimate the X-bar and Y-bar position parameters 
(column 12, lines 39-66). 

Claim 15, of curent application, compared to claim 28, of patent application, it 
disclose a method for augmenting reality of claim 13, wherein said step of scanning 
across the object to detect the fiducials includes the step of forming a signal 
corresponding to the detected pattern of fiducials, removing noise from the signal, and 
reducing the bandwidth of the signal (column 16, lines 25-29). 

Claim 33, of current application, compared to claim 1 1 , of patent application, it 
disclose a method according to claim 32, wherein said step of electronically reducing 
said virtual information comprises electronically filtering said information to eliminate 
background information. 
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Claim 34, of current application, compared to claim 12, of patent application, it 
disclose a method according to claim 32, wherein said step of electronically reducing 
said virtual infornnation comprises electronically filtering said infornoation to eliminate 
noise. 

Claim 35, of current application, compared to claim 13, of patent application, it 
disclose a method according to claim 34, wherein said step of electronically filtering said 
information to eliminate noise comprises electronically correcting to compensate for 
known aberrations of at least one hardware element. 

Claim Objections 

3. Claim 46 objected to under 37 CFR 1 .75(c), as being of improper dependent 
form for failing to further limit the subject matter of a previous claim. Applicant is 
required to cancel the clalm(s), or amend the clalm(s) to place the claim(s) in proper 
dependent form, or rewrite the claim(s) in independent form. Claim 1 12 does not exist. 

Claim Rejections - 35 USC $ 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in a patent granted on an application for patent by another filed in the 
United States before the invention thereof by the applicant for patent, or on an international application 
by another who has fulfilled the requirements of paragraphs (1 ), (2), and (4) of section 371 (c) of this 
title before the invention thereof by the applicant for patent. 
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5. Claims 1 -7, 9, 1 1 -1 6, 1 8-22, 25-39 and 41 -50, are rejected under 35 
U.S.C. 102(e) as being anticipated by Hirota et al (U.S. patent 6,064,749). 

Regarding claim 1 , Hirota discloses method for tracking the position and 
orientation of an object, comprising the steps of: 

(a) scanning across an object to detect fiducials, wherein a video run is formed 
by a scan (column 6, lines 30-36, generating images to locate landmarks within the field 
of view, also column 5, lines 45-52, tracking system and CCD video cameras 
(scanning); 

(b) clumping video runs to detect a pattern of fiducials (see column 3, line 13-25, 
refer to physical tracking camera position and orientation, and "fiducials" are utilized as 
the landmarks, also column 6, lines 50-54); 

(c) acquiring estimated values for a set of tracking parameters by comparing a 
detected pattern of fiducials to a reference pattern of fiducials (column 7, lines 20-40, 
the term fiducial refers to landmarks, also column 1 1 , lines 10-29, comparing between 
two predicted position); 

(d) iterating the estimated values for the set of tracking parameters until the 
detected pattem of fiducials match the reference pattern of fiducials to within a desired 
convergence (column 8, lines 58-64, compute and landmark predictor to iteratively 
detect additional landmarks, also column 16, lines 56-63 using iterative procedure). 

Regarding claim 2, Hirota discloses the method for tracking the position and 
orientation of an object of claim 1 , wherein the step of scanning across an object to 
detect a fiducials includes the step of setting a predetermined threshold voltage level for 
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detecting a fiducial, and Identifying fiducial edges when an output voltage from an 
optical sensor crosses the predetermined voltage level (column 9, lines 51-66, initial 
landmark location and second image based upon the predicted location and landmark 
located "within the defined search" region). 

Regarding claim 3, Hirota discloses the method for tracking the position and 
orientation of an object of claim 1 , wherein the step of clumping video runs includes the 
step of combining adjacent video runs and extracting relevant information from the 
video runs (column 17, lines 39-47, extracting information). 

Regarding claim 4, Hirota discloses the method for tracking the position and 
orientation of an object of claim 3, wherein a pixel is recorded by recording a scan line 
number and a pixel number for each pixel that has a video level above the 
predetermined threshold (column 1 1 , lines 53-60, refer to predefined range). 

Regarding claim 5, Hirota disclose the method for tracking the position and 
orientation of an object of claim 1 , wherein the step of clumping video runs includes the 
steps of detecting and removing noise from the video runs (column 17, lines 2-14, 
minimize the residual error). 

Regarding claim 6, Hirota discloses the method for tracking the position and 
orientation of an object of claim 1 , wherein the detected and reference patterns of 
fiducials includes a geometrically consistent pattern of hard fiducials and wherein the 
step of acquiring estimated values for a set of tracking parameters includes the step of 
corresponding a predetermined number of detected hard fiducials with the reference 
pattern of fiducials to estimate phi, theta and psi orientation parameters and to estimate 
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a distance position parameter (column 11, lines 50-62, number of marked pixels in both 
the inner dot and the outer ring are determined and ratio is computed. 

Regarding claim 7, Hirota discloses the method for tracking the position and 
orientation of an object of claim 1 , wherein the detected and reference patterns of 
fiducials includes a pseudo random pattern of soft fiducials and wherein the step of 
acquiring estimated values for a set of tracking parameters includes the step of electing 
at least one of the soft fiducials with the reference pattern of fiducials to estimate the X- 
bar and Y-bar position parameters (column 13, lines 37-40, landmarks are found on the 
two cameras' image planes, these landmarks provide 3 X-Y coordinate pairs). 

Regarding claim 9, Hirota et al disclose the method for tracking the position and 
orientation of an object of claim 1 , wherein the step of iterating the estimated values for 
the set of tracking parameters uses the method of least squares (column 15, lines 21- 
23, allows for the determination of an optimum solution using all the detected 
landmarks). 

Regarding claim 1 1 , Hirota discloses a method for augmenting reality, comprising 
steps of: (a) tracking the position and orientation of a pattem of fiducials on an object 
with a self-contained, mobile system (column 5, lines 50-59, view the augmented reality 
in the head mounted display, also column 9, lines 20-25); 

(b) processing virtual information stored in a computer memory of said system 
according to the position and orientation of the object (column 5, lines 26-37, refer to 
computer-readable storage): 
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(c) presenting the virtual information with real information to a user in near real 
time with said system (column 6, lines 7-16. transferring image in real-time). 

Regarding claim 12, Hirota discloses the method for augmenting reality of claim 
1 1 , wherein the pattern of fiducials are disposed on the object surface in a geometrically 
consistent hard fiducial pattern and in a pseudo random soft fiducial pattern (column 5, 
lines 67 through column 6, line 1-5). 

Regarding claim 14. Hirota discloses the method for augmenting reality of claim 
13, wherein the said step of scanning across the object to detect the fiducials includes 
the steps of setting a predetermined threshold voltage level for detecting a fiducial and 
identifying fiducial edges when an output voltage from an optical sensor crosses the 
predetermined voltage level (column 9, lines 51-66, initial landmark location and second 
image based upon the predicted location and landmark located "within the defined 
search" region). 

Regarding claim 15. Hirota discloses the method for augmenting reality of claim 
13, wherein said step of scanning across the object to detect the fiducials includes the 
step of forming a signal corresponding to the detected pattern of fiducials, removing 
noise from the signal, and reducing the bandwidth of the signal (column 7, lines 21-34, . 
error correcting (removing noise), transformation); 

and electing at least one of the soft fiducials with the reference pattern of 
fiducials to estimate X-bar and Y-bar position parameters (column 13, lines 37-40, 
landmarks are found on the two cameras' image planes, these landmarks provide 3 X-Y 
coordinate pairs). 
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Regarding claim 18, Hirota discloses the method for augmenting reality of claim 
1 1 , wherein the computer memory forms a part of a wearable computer, the wearable 
computer having processor which performs said step of processing virtual infornfiation 
stored in the computer memory according to position and orientation of the object 
(column 6, lines 7-16). 

Regarding claim 19, Hirota discloses the method for augmenting reality of claim 
1 1 , wherein said step of presenting the virtual infonnation with real information to a user 
in near real time includes the step of projecting the virtual information on a head 
mounted display (column 6, lines 7-16, computer generated graphics, and composite 
images are then display). 

Regarding claim 20, Hirota discloses the method for augmenting reality of claim 
11, wherein said step of presenting the virtual information with real information to a user 
in near real time includes the step of projecting the virtual information on an Optical See 
Through display (column 6, lines 7-16, transferring image in real-time). 

Regarding claim 21 , Hirota discloses the method for augmenting reality of claim 
11, wherein said step of presenting the virtual information with real information to a user 
in near real time includes the step of projecting the virtual information and the real 
information on a Video See Through display (column 6, lines 40-49, computer 
generated graphics are registered to object in the real word). 

Regarding claim 22, Hirota discloses the method for augmenting reality of claim 
11, wherein said steps of tracking the position and orientation of the object, retrieving 
virtual information, and presenting the virtual information with real information are 
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performed with an update rate of at least 60 Hertz and a latency below 16 milliseconds 
(column 9, lines 4-13). 

Regarding claim 25, Hirota discloses the method for augmenting reality of claim 
1 1 , further comprising the step of calibrating the alignment between the virtual 
information and the real information (column 7, lines 1-4). 

Regarding claim 27, Hirota discloses a method according to claim 26, wherein 
step (d) comprises providing computer-generated aural information to a user (column 5, 
lines 53-58, head mounted unit is worn by a user for communication, also column 7, 
lines 41-48, refer to stereo cameras). 

Regarding claim 28, Hirota discloses a method according to claim 26, wherein 
step (d) comprises providing computer-generated kinesthetic infonnation to a user 
(column 6, lines 27-42, refer to background (kinesthetic)). 

Regarding claim 29, Hirota discloses a method according to claim 26, wherein 
step (a) comprises using a sensor to provide at least one signal that is indicative of a 
pattern of fiducials on a user's eye (column 5, lines 50-59, user which views the 
augmented). 

Regarding claim 30, Hirota discloses a method according to claim 26, wherein 
said sensor is mounted on a user's head (column 5, lines 50-59, refer to head mounted 
display). 

Regarding claim 32, Hirota discloses a method according to claim 26, wherein at 
least one of steps (b) and (c) are performed by electronically reducing said virtual 
infornnation to a reduced amount of information for manageable processing (column 1 1 , 
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lines 31-49, innage analyzer 42, reduces the search areas while refining its estiniate of 
the landmark's location). 

Regarding claim 37, Hirota discloses a method according to daim 32, wherein 
said step of electronically reducing said virtual information comprises a step of 
determining a centroid position of at least one of said fiducials (column 1 1 , line 53 
through column 12, line 9, refer to center, inner and outer area). 

Regarding claim 49, Hirota discloses a method according to claim 48, wherein 
said quick acquire routine tracks movement of the system with respect to the object and 
utilizes data pertaining to said movement to predict likely future locations of said fiducial 
(column 5, lines 39-59, movement and tracking system). 

Regarding claims 13, 16, the arguments analogous to those presented for claims 
1, 6, are applicable. 

Regarding claims 26 and 31 , the arguments analogous to those presented for 
claims 1.11 and 18 are applicable. 

Regarding claim 33-36 and 38-39, the arguments analogous to those presented 
for claims 1 , 5, 10 and 37 are applicable. 

Regarding claims 41-46, the arguments analogous to those presented for claims 
1 , 6 and 7 are applicable. 

Regarding claims 47-48 and 50, the arguments analogous to those presented for 
claims 1 and 27-28 are applicable. 



Application/Control Number: 10/664,565 Page 12 

Art Unit: 2625 

Claim Rejections ■ 35 USC S 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claim 17, is rejected under 35 U.S.C. 103(a) as being unpatentable over Hirota 
et al (U.S. patent 6,064,749) in view of Carollo (U.S. patent 5,912,650). 

Regarding claim 17, Hirota do not explicitly state, "process of fabricating wire 
harnesses". On the other hand Carollo teaches column 5, line 20-26, providing the user 
with an augmented reality with additional information regarding a scene under view 
such as wiring harness. 

Therefore it would have been obvious to one having ordinary skill in the art at the 
time of the invention was made, to modify Hirota et al invention according to the 
teaching of Carollo because it combined with a see-through display, to providing the 
user with an augmented reality. This is particularly useful for night vision application or 
when it is useful to provide the user with additional information regarding a scene under 
view. 

8. Claim 23-24, are rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Hirota et al (US patent 6,064,749) in view of DeMenthon (U.S. patent 5,227,985). 

Regarding claim 23, Hirota do not explicitly state, "user selects menu item by 
moving a cursor over a desired selection". On the other hand DeMenthon teaches 
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(column 6, lines 16-29, a virtual cursor of known 3D structure is considered to be rigidly 
linked to the object and therefore move along with object, also column 18, lines 9-27, 
screen cursor 94 is the perspective projection of a virtual 3D cursor of known 3D 
structure). 

Therefore it would have been obvious to one having ordinary skill in the art at the 
time of the invention was made, to modify Hirota et al invention according to the 
teaching of DeMenthon because it provides a tool to interact with the other virtual 
objects of the scenery, which can easily be implemented in a image device such as 
video camera. 

Regarding claim 24, the arguments analogous to those presented for claim 23 are 
applicable. 

Allowable Subject Matter 

9. Claims 8, 10 and 40, objected to as being dependent upon a rejected base claim, 
but would be allowable if rewritten in independent form including all of the limitations of 
the base claim and any intervening claims. 

Other prior art cited 

1 0. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

U.S. patent (U.S.5,41 2,569) to Corby, Jr. et al is cited for augmented reality 
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maintenance system witfi archive and comparison device. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta, can be reached at (571) 272-7453. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 

Patent Application infornnation Retrieval (PAIR) system. Status information for published 
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